Hybrid Batch Bayesian Optimization

Bayesian Optimization (BO) aims at optimizing an unknown function that is costly to evaluate. We focus on applications where concurrent function evaluations are possible. In such cases, BO could choose to either sequentially evaluate the function (sequential mode) or evaluate the function at a batch of multiple inputs at once (batch mode). The sequential mode generally leads to better optimization performance as each function evaluation is selected with more information, whereas the batch mode is more time efficient (smaller number of iterations). Our goal is to combine the strength of both settings. We systematically analyze BO using a Gaussian Process as the posterior estimator and provide a hybrid algorithm that dynamically switches between sequential and batch with variable batch sizes. We theoretically justify our algorithm and present experimental results on eight benchmark BO problems. The results show that our method achieves substantial speedup (up to 78%) compared to sequential, without suffering any significant performance loss.